Statistical Tests Using Hinge/ε-Sensitive Loss
نویسندگان
چکیده
Abstract. Statistical tests used in the literature to compare algorithms use the misclassification error which is based on the 0/1 loss and square loss for regression. Kernel-based, support vector machine classifiers (regressors) however are trained to minimize the hinge ( -sensitive) loss and hence they should not be assessed or compared in terms of the 0/1 (square loss) but with the loss measure they are trained to minimize. We discuss how the paired t test can use the hinge ( -sensitive) loss and show in our experiments that doing that, we can detect differences that the test on error cannot detect, indicating higher power in distinguishing between the behavior of kernel-based classifiers (regressors). Such tests can be generalized to compare L > 2 algorithms.
منابع مشابه
Lower Bounds for BMRM and Faster Rates for Training SVMs
Regularized risk minimization with the binary hinge loss and its variants lies at the heart of many machine learning problems. Bundle methods for regularized risk minimization (BMRM) and the closely related SVMStruct are considered the best general purpose solvers to tackle this problem. It was recently shown that BMRM requires O(1/ε) iterations to converge to an ε accurate solution. In the fir...
متن کاملSmooth ε-Insensitive Regression by Loss Symmetrization
We describe a framework for solving regression problems by reduction to classification. Our reduction is based on symmetrization of margin-based loss functions commonly used in boosting algorithms, namely, the logistic-loss and the exponential-loss. Our construction yields a smooth version of the ε-insensitive hinge loss that is used in support vector regression. Furthermore, this construction ...
متن کاملSemi-supervised Learning using Kernel Self-consistent Labeling
We present a new method for semi-supervised learning based on any given valid kernel. Our strategy is to view the kernel as the covariance matrix of a Gaussian process and predict the label of each instance conditioned on all other instances. We then find a self-consistent labeling of the instances by using the hinge loss on the predictions on labeled data and the ε insensitive loss on predicti...
متن کاملSimple Risk Bounds for Position-Sensitive Max-Margin Ranking Algorithms
R bounds for position-sensitive max-margin ranking algorithms can be derived straightforwardly from a structural result for Rademacher averages presented by [1]. We apply this result to pairwise and listwise hinge loss that are position-sensitive by virtue of rescaling the margin by a pairwise or listwise position-sensitive prediction loss. Similar bounds have recently been presented for probab...
متن کاملScalable Support Vector Machine for Semi-supervised Learning
Owing to the prevalence of unlabeled data, semisupervised learning has recently drawn significant attention and has found applicable in many real-world applications. In this paper, we present the so-called Graph-based Semi-supervised Support Vector Machine (gS3VM), a method that leverages the excellent generalization ability of kernel-based method with the geometrical and distributive informati...
متن کامل